Method and apparatus for facilitating interaction with an object viewable via a display

ABSTRACT

A method, apparatus and computer program product are provided to facilitate interaction with a computing device, such as interaction with an object viewable via a display of a computing device. In the context of a method, an object viewable via a display is identified, such as by facial recognition. The method also includes determining that the object has become aligned with an operation indicator as a result of movement of the display relative to the object. In response to determining that the object is aligned with the operation indicator, the method also performs an operation associated with the operation indicator. In this regard, a method may perform the operation upon at least one aspect associated with the object. Corresponding apparatus and computer program products are also provided.

TECHNOLOGICAL FIELD

An example embodiment of the present invention relates generally to amethod and apparatus for facilitating interaction with a computingdevice and, more particularly, to a method and apparatus for interactingwith an object viewable via a display of the computing device.

BACKGROUND

Users of computing devices interact with those devices on a frequent andrepeated basis. Accordingly, a number of different techniques have beendeveloped to facilitate interaction with the computing devices. Thesetechniques may include the receipt of user input via a keypad, via atouchscreen or via audible commands, to name but a few. Although varioustechniques for facilitating user interaction have been developed, theuser experience may be improved by further improvements related to userinteraction with computing devices.

By way of example, pass-through displays may permit a user to view oneor more objects through the display. Similarly, other types of computingdevices may present an image upon the display, such as an image capturedby a camera carried by the computing device, with the image includingone or more objects. In either instance, a user may wish to interactwith an object that is viewable via the display, such as by performingan operation upon the object. The user experience may therefore beimproved by correspondingly providing further improvements in regards tothe user's interaction with objects viewable via the display of acomputing device.

BRIEF SUMMARY

A method, apparatus and computer program product are therefore providedaccording to an example embodiment of the present invention in order tofacilitate interaction with a computing device, such as interaction withan object viewable via a display of a computing device. In this regard,the method, apparatus and computer program product of an exampleembodiment may permit an operation to be performed upon at least oneaspect associated with an object viewable via the display. As such, theuser experience with a computing device, such as a pass-through display,may be improved as a result of the interaction provided in accordancewith a method, apparatus and computer program product of an exampleembodiment.

In one embodiment, a method is provided that includes identifying anobject viewable via a display. For example, the method of one embodimentmay employ facial recognition to identify a person. The method alsoincludes determining, with a processor, that the object has becomealigned with an operation indicator as a result of movement of thedisplay relative to the object. In response to determining that theobject is aligned with the operation indicator, the method also performsan operation associated with the operation indicator. In this regard, amethod may perform the operation upon at least one aspect associatedwith the object.

The method of one embodiment may also include providing for selection ofthe object. In this embodiment, the performance of the operation may notonly be dependent upon the determination that the object is aligned withthe operation indicator, but also upon the selection of the object. Inone embodiment, the method also causes the operation indicator to bepresented upon the display. In an embodiment in which the displaycomprises a pass-through display, the method may identify the object byidentifying an object viewable through the pass-through display. Inanother embodiment, the method may receive an image that includes theobject and may cause the image to be presented upon the display prior toidentifying the object.

In another embodiment, an apparatus is provided that includes at leastone processor and at least one memory including computer program codewith at least one memory and computer program code configured to, withthe processor, cause the apparatus to at least identify an objectviewable via a display, such as by employing facial recognition toidentify a person. The at least one memory and the computer program codealso configured to, with the processor, cause the apparatus to determinethat the object has become aligned with an operation indicator as aresult of movement of the display relative to the object. The at leastone memory and the computer program code are also configured to, withthe processor, cause the apparatus of this embodiment to perform anoperation associated with the operation indicator in response todetermining that the object is aligned with the operation indicator. Theat least one memory and the computer program code are also configuredto, with the processor, cause the apparatus of this embodiment toperform the operation upon at least one aspect associated with theobject.

The at least one memory and the computer program code may be furtherconfigured to, with the processor, cause the apparatus of an exampleembodiment to provide for selection of the object. In this embodiment,the performance of the operation may not only be dependent upon thedetermination that the object is aligned with the operation indicator,but also upon the selection of the object. In one embodiment, the atleast one memory and the computer program code may also be configuredto, with the processor, cause the apparatus to cause the operationindicator to be presented upon the display. In an embodiment in whichthe display includes a pass-through display, the at least one memory andthe computer program code may be configured to, with the processor,cause the apparatus to identify an object by identifying an objectviewable through the pass-through display. The at least one memory andthe computer program code may be further configured to, with theprocessor, cause the apparatus of another embodiment to receive an imagethat includes the object and to cause the image to be presented upon thedisplay prior to identifying the object.

In a further embodiment, a computer program product is provided thatincludes at least one non-transitory computer-readable storage mediumhaving computer-executable program code portions stored therein with thecomputer-executable program code portions including program codeinstructions for identifying objects viewable via a display, such as byemploying facial recognition to identify a person. Thecomputer-executable program code versions of this embodiment alsoinclude program code instructions for determining that the object hasbecome aligned with an operation indicator as a result of movement ofthe display relative to the object. The computer-executable program codeportions of this embodiment also include program code instructions forperforming an operation associated with the operation indicator inresponse to determining that the object is aligned with the operationindicator. In this regard, the program code instructions for performingthe operation may include program code instructions for performing theoperation based upon at least one aspect associated with the object.

The computer-executable program code portions of one embodiment may alsoinclude program code instructions for providing for selection of theobject. In this regard, the program code instructions for performing theoperation may not only be dependent upon a determination that the objectis aligned with the operation indicator, but may also require the objectto have been selected prior to performing the operation. In oneembodiment, the computer-executable program code portion may alsoinclude program code instructions for causing the operation indicator tobe presented upon the display. In an embodiment in which the displayincludes a pass-through display, the program code instructions foridentifying an object may include program code instructions foridentifying an object viewable through the pass-through display. Inanother embodiment, the computer-executable program code portions mayalso include program code instructions for receiving the image thatincludes the object and causing the image to be presented upon thedisplay prior to identifying the object.

In yet another embodiment, an apparatus is provided that includes meansfor identifying an object viewable via a display. The apparatus of thisembodiment may also include means for determining that the object hasbecome aligned with an operation indicator as a result of movement ofthe display relative to the object. The apparatus of this embodiment mayalso include means for performing an operation associated with theoperation indicator in response to determining that the object isaligned with the operation indicator. In this regard, the means forperforming the operation may be configured to perform the operation uponat least one aspect associated with the object.

BRIEF DESCRIPTION OF THE DRAWINGS

Having thus described certain example embodiments of the presentinvention in general terms, reference will hereinafter be made to theaccompanying drawings, which are not necessarily drawn to scale, andwherein:

FIG. 1 is block diagram of an apparatus that may be specificallyconfigured in accordance with an example embodiment of the presentinvention;

FIG. 2 is perspective view of a pass-through display that may bespecifically configured in accordance with an example embodiment of thepresent invention;

FIG. 3 is a plan view of a mobile terminal that may be specificallyconfigured in accordance with an example embodiment to the presentinvention;

FIG. 4 is a flowchart illustrating operations performed, such as by theapparatus of FIG. 1, in accordance with an example embodiment to thepresent invention;

FIG. 5 is an image that includes an object, a plurality of operationindicators and a representation of a cursor;

FIG. 6 is the image of FIG. 5 with one of the operation indicatorshaving become aligned with the object as a result of movement of thedisplay relative to the object and in accordance with an exampleembodiment of the present invention; and

FIG. 7 is the image of FIGS. 5 and 6 which depicts the performance of anoperation upon at least one aspect associated with the object inresponse to determining that the object is aligned with the operationindicator in accordance with an example embodiment to the presentinvention.

DETAILED DESCRIPTION

Some embodiments of the present invention will now be described morefully hereinafter with reference to the accompanying drawings, in whichsome, but not all, embodiments of the invention are shown. Indeed,various embodiments of the invention may be embodied in many differentforms and should not be construed as limited to the embodiments setforth herein; rather, these embodiments are provided so that thisdisclosure will satisfy applicable legal requirements. Like referencenumerals refer to like elements throughout. As used herein, the terms“data,” “content,” “information,” and similar terms may be usedinterchangeably to refer to data capable of being transmitted, receivedand/or stored in accordance with embodiments of the present invention.Thus, use of any such terms should not be taken to limit the spirit andscope of embodiments of the present invention.

Additionally, as used herein, the term ‘circuitry’ refers to (a)hardware-only circuit implementations (e.g., implementations in analogcircuitry and/or digital circuitry); (b) combinations of circuits andcomputer program product(s) comprising software and/or firmwareinstructions stored on one or more computer readable memories that worktogether to cause an apparatus to perform one or more functionsdescribed herein; and (c) circuits, such as, for example, amicroprocessor(s) or a portion of a microprocessor(s), that requiresoftware or firmware for operation even if the software or firmware isnot physically present. This definition of ‘circuitry’ applies to alluses of this term herein, including in any claims. As a further example,as used herein, the term ‘circuitry’ also includes an implementationcomprising one or more processors and/or portion(s) thereof andaccompanying software and/or firmware. As another example, the term‘circuitry’ as used herein also includes, for example, a basebandintegrated circuit or applications processor integrated circuit for amobile phone or a similar integrated circuit in a server, a cellularnetwork device, other network device, and/or other computing device.

As defined herein, a “computer-readable storage medium,” which refers toa non-transitory physical storage medium (e.g., volatile or non-volatilememory device), can be differentiated from a “computer-readabletransmission medium,” which refers to an electromagnetic signal.

A method, apparatus and computer program product are provided accordingto an example embodiment of the present invention in order to facilitateuser interaction with objects viewable via a display. In this regard,the method, apparatus and computer program product of an exampleembodiment may permit an operation to be performed upon at least oneaspect associated with the object. Although the method, apparatus andcomputer program product may be utilized in conjunction with a varietyof computing devices having associated displays upon which an object isviewable, the method, apparatus and computer program product of oneexample embodiment may facilitate user interaction with an objectviewable via a pass-through display such that an operation may beperformed upon at least one aspect associated with the object viewablethrough the pass-through display. The method, apparatus and computerprogram product of this embodiment may therefore permit the user tointeract more readily with objects in their vicinity.

An example embodiment of the invention will now be described withreference to FIG. 1, in which certain elements of an apparatus 10 forfacilitating interaction with an object viewable via a display aredepicted. The apparatus of FIG. 1 may be employed, for example, inconjunction with, such as by being incorporated into or embodied by,various computing devices, such as a pass-through display or a mobileterminal. However, it should be noted that the apparatus of FIG. 1 mayalso be employed in connection with a variety of other computing devicesand therefore, embodiments of the present invention should not belimited to application with the various examples of the computingdevices that are described below.

It should also be noted that while FIG. 1 illustrates one example of aconfiguration of an apparatus 10 for facilitating interaction with anobject viewable via a display, numerous other configurations may also beused to implement embodiments of the present invention. As such, in someembodiments, although devices or elements are shown as being incommunication with each other, hereinafter such devices or elementsshould be considered to be capable of being embodied within the samedevice or element and thus, devices or elements shown in communicationshould be understood to alternatively be portions of the same device orelement.

Referring now to FIG. 1, the apparatus 10 for facilitating interactionwith an object viewable via a display may include or otherwise be incommunication with a processor 12, a memory device 14, a communicationinterface 16 and optionally a user interface 18 and a camera 20. In thisregard, reference to a camera includes an embodiment that includes asingle camera as well as an embodiment that includes a plurality ofcameras, such as to facilitate construction of a wide angle image. Insome embodiments, the processor (and/or co-processors or any otherprocessing circuitry assisting or otherwise associated with theprocessor) may be in communication with the memory device via a bus forpassing information among components of the apparatus. The memory devicemay be non-transitory and may include, for example, one or more volatileand/or non-volatile memories. In other words, for example, the memorydevice may be an electronic storage device (e.g., a computer readablestorage medium) comprising gates configured to store data (e.g., bits)that may be retrievable by a machine (e.g., a computing device like theprocessor). The memory device may be configured to store information,data, content, applications, instructions, or the like for enabling theapparatus to carry out various functions in accordance with an exampleembodiment of the present invention. For example, the memory devicecould be configured to buffer input data for processing by theprocessor. Additionally or alternatively, the memory device could beconfigured to store instructions for execution by the processor.

The apparatus 10 may be embodied by a computing device, such as apass-through display or a mobile terminal. However, in some embodiments,the apparatus may be embodied as a chip or chip set. In other words, theapparatus may comprise one or more physical packages (e.g., chips)including materials, components and/or wires on a structural assembly(e.g., a baseboard). The structural assembly may provide physicalstrength, conservation of size, and/or limitation of electricalinteraction for component circuitry included thereon. The apparatus maytherefore, in some cases, be configured to implement an embodiment ofthe present invention on a single chip or as a single “system on achip.” As such, in some cases, a chip or chipset may constitute meansfor performing one or more operations for providing the functionalitiesdescribed herein.

The processor 12 may be embodied in a number of different ways. Forexample, the processor may be embodied as one or more of varioushardware processing means such as a coprocessor, a microprocessor, acontroller, a digital signal processor (DSP), a processing element withor without an accompanying DSP, or various other processing circuitryincluding integrated circuits such as, for example, an ASIC (applicationspecific integrated circuit), an FPGA (field programmable gate array), amicrocontroller unit (MCU), a hardware accelerator, a special-purposecomputer chip, or the like. As such, in some embodiments, the processormay include one or more processing cores configured to performindependently. A multi-core processor may enable multiprocessing withina single physical package. Additionally or alternatively, the processormay include one or more processors configured in tandem via the bus toenable independent execution of instructions, pipelining and/ormultithreading.

In an example embodiment, the processor 12 may be configured to executeinstructions stored in the memory device 14 or otherwise accessible tothe processor. Alternatively or additionally, the processor may beconfigured to execute hard coded functionality. As such, whetherconfigured by hardware or software methods, or by a combination thereof,the processor may represent an entity (e.g., physically embodied incircuitry) capable of performing operations according to an embodimentof the present invention while configured accordingly. Thus, forexample, when the processor is embodied as an ASIC, FPGA or the like,the processor may be specifically configured hardware for conducting theoperations described herein. Alternatively, as another example, when theprocessor is embodied as an executor of software instructions, theinstructions may specifically configure the processor to perform thealgorithms and/or operations described herein when the instructions areexecuted. However, in some cases, the processor may be a processor of aspecific device (e.g., a pass-through display or a mobile terminal)configured to employ an embodiment of the present invention by furtherconfiguration of the processor by instructions for performing thealgorithms and/or operations described herein. The processor mayinclude, among other things, a clock, an arithmetic logic unit (ALU) andlogic gates configured to support operation of the processor.

Meanwhile, the communication interface 16 may be any means such as adevice or circuitry embodied in either hardware or a combination ofhardware and software that is configured to receive and/or transmit datafrom/to a network and/or any other device or module in communicationwith the apparatus 10. In this regard, the communication interface mayinclude, for example, an antenna (or multiple antennas) and supportinghardware and/or software for enabling communications with a wirelesscommunication network. Additionally or alternatively, the communicationinterface may include the circuitry for interacting with the antenna(s)to cause transmission of signals via the antenna(s) or to handle receiptof signals received via the antenna(s). In some environments, thecommunication interface may alternatively or also support wiredcommunication. As such, for example, the communication interface mayinclude a communication modem and/or other hardware/software forsupporting communication via cable, digital subscriber line (DSL),universal serial bus (USB) or other mechanisms.

In some embodiments, the apparatus 10 may include a user interface 18that may, in turn, be in communication with the processor 12 to provideoutput to the user and, in some embodiments, to receive an indication ofa user input. As such, the user interface may include a display and, insome embodiments, may also include a keyboard, a mouse, a joystick, atouch screen, touch areas, soft keys, a microphone, a speaker, or otherinput/output mechanisms. Alternatively or additionally, the processormay comprise user interface circuitry configured to control at leastsome functions of one or more user interface elements such as a displayand, in some embodiments, a speaker, ringer, microphone and/or the like.The processor and/or user interface circuitry comprising the processormay be configured to control one or more functions of one or more userinterface elements through computer program instructions (e.g., softwareand/or firmware) stored on a memory accessible to the processor (e.g.,memory device 14, and/or the like).

In some example embodiments, the apparatus 10 may include an imagecapturing element, such as a camera 20, video and/or audio module, incommunication with the processor 12. The image capturing element may beany means for capturing an image, video and/or audio for storage,display or transmission. As used herein, an image includes a still imageas well as an image from a video recording. For example, in an exampleembodiment in which the image capturing element is a camera, the cameramay include a digital camera capable of forming a digital image filefrom a captured image. As such, the camera may include all hardware (forexample, a lens or other optical component(s), image sensor, imagesignal processor, and/or the like) and software necessary for creating adigital image file from a captured image. Alternatively, the camera mayinclude only the hardware needed to view an image, while the memorydevice 14 of the apparatus stores instructions for execution by theprocessor in the form of software necessary to create a digital imagefile from a captured image. In an example embodiment, the camera mayfurther include a processing element such as a co-processor whichassists the processor in processing image data and an encoder and/ordecoder for compressing and/or decompressing image data. The encoderand/or decoder may encode and/or decode according to, for example, ajoint photographic experts group (JPEG) standard, a moving pictureexperts group (MPEG) standard, or other format.

The apparatus 10 of FIG. 1 may be embodied by or otherwise associatedwith a variety of different computing devices, each of which has anassociated display. For example, the apparatus may be embodied by orassociated with a pass-through display. A pass-through display may beembodied in various manners. For example, the pass-through display maybe a near-eye display, such as a head worn display, through which theuser may optically view a scene external to the near-eye display. By wayof example, a near-eye display of one embodiment is shown in FIG. 2 inthe form of a pair of eyeglasses 30. The eyeglasses may be worn by auser such that the user may view a scene, e.g., a field of view, throughthe lenses of the eyeglasses. However, the eyeglasses of this embodimentmay also be configured to present a visual representation of informationupon the lenses so as to augment or supplement the user's view of thescene through the lenses of the eyeglasses. The information presented bythe eyeglasses may augment the objects in the scene viewed through theeyeglasses, such as by identifying or otherwise providing moreinformation regarding one or more of the objects viewed through theeyeglasses. Alternatively, the information presented by the eyeglassesmay be unrelated to the objects in the scene viewed through theeyeglasses, but may otherwise provide information that may be ofinterest to the user. Regardless of the type of information presented bythe eyeglasses, a pass-through display as exemplified by the eyeglassesof FIG. 2 may support augmented reality and other applications. Asanother example, the pass-through display may be embodied by awindshield, a visor, a helmet or other type of display through which auser optically views an image or a scene external to the display. Whileexamples of a pass-through display have been provided, a pass-throughdisplay may be embodied in a number of different manners with a varietyof form factors, each of which may permit a user to optically seethrough the display so as to view the user's surroundings and each ofwhich may benefit from the method, apparatus and computer programproduct of an example embodiment of the present invention as describedbelow.

By way of another example, the apparatus 10 of FIG. 1 may be embodied byor associated with a mobile terminal 40. The mobile terminal may beembodied by any of a variety of computing devices including, but notlimited to, a portable digital assistant (PDA), mobile telephone,smartphone, pager, mobile television, gaming device, laptop computer,camera, tablet computer, touch surface, video recorder, audio/videoplayer, radio, electronic book, positioning device (e.g., globalpositioning system (GPS) device), or any combination of theaforementioned, and other types of voice and text communicationssystems. Regardless of the type of computing device, the mobile terminalmay include a display 42 and a user interface. In the illustratedembodiment, the user interface includes such as one or more keys 48 forreceiving user input. However, the mobile terminal need not include thekeys and may, instead, include another type of user interface, such as atouch screen display. The apparatus embodied by or associated with themobile terminal, such as shown in FIG. 3, may include a camera 20 so asto capture an image, such as an image of the surroundings proximate themobile terminal. The image may, in turn, be presented upon the display,such as the image of a woman holding a pair of books that is presentedupon the display of the embodiment of the mobile terminal in FIG. 3.

In one embodiment, the apparatus 10 may be embodied by the computingdevice, such as the pass-through display or the mobile terminal 30.Alternatively, the apparatus may be embodied by a computing device thatis remote from the display upon which the object is viewable, but thatis in communication therewith, such as via wireless communication, e.g.,via Bluetooth communication, Wi-Fi or another wireless network, or viawired communication, utilizing the communication interface 16.

Regardless of the embodiment, an object is viewable via a display of acomputing device, such as the computing device that embodies or isotherwise associated with the apparatus 10 of FIG. 1. With respect to apass-through display, an object may be viewable through the display,such as an object proximate the user at which the user is looking.Alternatively, in an embodiment in which a mobile terminal 30 includesor is otherwise associated with a camera that captures an image, theimage captured by the camera, such as an image of the user'ssurroundings, may be presented upon the display such that a user mayview an object within the image presented upon the display. One exampleof such an image is presented in FIG. 3. As such, an object viewed by auser may be viewable through the display or upon the display.

The method, apparatus 10 and computer program product of an exampleembodiment will now be described in conjunction with the operationsillustrated in FIG. 4. In this regard, the apparatus 10 may includemeans, such as the processor 12, the user interface 18 or the like, foridentifying an object viewable via the display. See block 52 of FIG. 4.The object may be identified in various manners. In one embodiment inwhich the object is a person, the apparatus, such as the processor, maybe configured to employ facial recognition in order to identify theperson. In this regard, FIG. 5 depicts an image that is viewable via adisplay, such as an image viewable through a pass-through display or animage presented upon the display of a mobile terminal. In the embodimentof FIG. 5, the object is a person, who has been recognized by facialrecognition as indicated by the box that bounds the person's face.

Alternatively, in an instance in which the object is a building,landmark or other structure, the apparatus 10, such as the processor 12,may be configured to identify the object based upon the geographiclocation of the object, such as may be estimated by the geographiclocation of the computing device as provided, for example, by apositioning sensor, and with reference to a database, such as may bemaintained by memory 14, which identifies various buildings, landmarksor other structures by geographic location. In this embodiment, thepositioning sensor may include, for example, a global positioning system(GPS) sensor, an assisted global positioning system (A-GPS) sensor, aBluetooth (BT)-GPS mouse, other GPS or positioning receivers or thelike. The positioning sensor may be capable of determining a location ofthe computing device that embodies the apparatus, such as, for example,longitudinal and latitudinal directions of the computing device, or aposition relative to a reference point such as a destination or startpoint.

Alternatively, the apparatus 10, such as the processor 12, may beconfigured to identify a building, landmark or other structure bycomparing the appearance of the building, landmark or other structurethat is viewable via the display with a database, such as may bemaintained by memory 14, of images of various buildings, landmarks andother structures in order to identify a building, landmark or otherstructure that has a matching appearance. The apparatus, such as theprocessor, the user interface 18 or the like, may be configured toidentify the object viewable via the display in other manners. Forexample, the apparatus, such as the processor, the user interface or thelike, may be configured to identify the object by permitting the user toprovide an identification of the object, such as by entering the name ofthe object or other indicia identifying the object.

Regardless of the manner in which the object is identified, theapparatus 10, such as the processor 12, the user interface 18 or thelike, may cause a representation of the identification of the object,such as the name of the object or other indicia identifying the object,to be presented upon the display in proximity to the object. Forexample, the person that was identified in the embodiment of FIG. 5 isidentified as Lily Wang.

As shown in FIGS. 2, 3 and 5, the apparatus 10 of one embodiment mayinclude means, such as the processor 12, the user interface 18 or thelike, for causing one or more operation indicators to be presented uponthe display. See block 50 of FIG. 4. In this regard, the circles 32, 44,60 that are presented upon the displays of FIGS. 2, 3 and 5,respectively, are representative of operation indicators. Operationindicators may take various forms, such as icons, alpha-numericdesignations or the like. Regardless of the manner in which an operationindicator is presented, an operation indicator is representative of anoperation that may be performed upon selection of the operationindicator. By way of example, a short message service (SMS) operationindicator, as shown in FIG. 5, may cause an SMS message to be generatedupon selection of the SMS operation indicator. As another example, aFacebook operation indicator may cause a Facebook post to be generatedupon actuation of the Facebook operation indicator. As a furtherexample, a camera operation indicator may cause the image viewable viathe display to be captured upon actuation of the camera operationindicator. Additionally, a gallery operation indicator may cause allrelevant pictures and/or videos that include a particular person to belisted upon actuation of the gallery operation indicator. An outlookoperation indicator may cause an electronic mail message to be generatedupon selection of the outlook operation indicator and, in some instancesmay also cause one or more history dialogues with a particular person tobe shown. Further, a social networking operation indicator may cause toa listing of the common friends between the user and a particular personto be shown upon selection of the social networking operation indicator.While certain examples of an operation indicator are provided above,there may be many other types of operation indicators with the foregoingprovided by way of example, but not of limitation.

As shown in FIGS. 3 and 5, the operation indicators 44, 60 may be offsetfrom the object that is viewable via the display. In this regard, theobject indicators may not be aligned with or overlap the object, but,instead, may be spaced apart from the object. However, the apparatus ofone embodiment includes means, such as the processor 12 or the like, forthereafter determining that the object has become aligned with theoperation indicator. See block 54 of FIG. 4. In this regard, the objectis aligned with the operation indicator in an instance in which theoperation indicator overlies, either fully or at least partially, theobject or a designated region of the object. In this regard, FIG. 5depicts a scene or an image that is viewable via the display, e.g., ascene viewable through a pass-through display or an image presented uponthe display of a mobile terminal. As described above, the object hasbeen identified, such as by facial recognition, as Lily Wang. Inaddition, the two operation indicators 60 are offset from and thereforeare not aligned with the object. In FIG. 6, however, the SMS operationindicator is in alignment with the object and, more particularly, with adesignated portion of the object, that is, the face of Lily Wang thatwas the subject of facial recognition as designated by the box boundingher face. Thus, in an instance in which a portion of the object viewablevia the display is designated, such as the face of a person that wasidentified via facial recognition as designated by a bounding box, thealignment of an operation indicator with the object may be accomplishedby aligning the operation indicator with the designated portion of theobject. Alternatively, in an instance in which no portion of the objectviewable via the display is specifically designated, the alignment ofthe operation indicator with the object may be accomplished by aligningthe operation indicator with any portion of the object.

In accordance with an example embodiment to the present invention, theobject may be aligned with the operation indicator as a result ofmovement of the display relative to the object as opposed to, forexample, movement of the operation indicator, such as the result of adrag operation, relative to the display. For example, in an instance inwhich the display is a pass-through display, the user wearing thepass-through display, such as the user wearing the eyeglasses 30 of FIG.2, may move their head in such a manner that the object viewable via thedisplay becomes aligned with the operation indicator 32. Alternatively,in an instance in which the display is the display of a mobile terminal40 as shown in FIG. 3, the mobile terminal may be moved or repositionedsuch that the image captured by the camera 20 following therepositioning or the mobile terminal includes the object in a positionthat is now aligned with the operation indicator. As will be noted bythe foregoing examples, the alignment of the object with respect to theoperation indicator is brought about as a result of movement of thedisplay, such as the eyeglasses of FIG. 2 or the mobile terminal of FIG.3, relative to the object and the relative position of the operationindicators upon the display remain the same, that is, the relativeposition of the operation indicators upon the display are independent ofmovement of the display relative to the object. As such, the relativeposition of object with respect with respect to the display is changedas a result of the relative movement of the display with respect to theobject as exemplified by a comparison of FIGS. 5 and 6, withoutrepositioning the operation indicators upon the display.

The apparatus 10 may also include means, such as the processor 12, theuser interface 18 or the like, for performing an operation associatedwith the operation indicator in response to determining that the objectis aligned with the operation indicator. See block 58 of FIG. 4. In thisregard, the apparatus, such as the processor, may be configured toperform the operation upon at least one aspect associated with theobject. The operations that are performed in response to determiningthat the object is aligned with the operation indicator depends upon theoperation indicator with which the object has become aligned. In regardsto the SMS operation indicator, a SMS message may be generated as shownin FIG. 7. Alternatively, the operation performed in response to theobject being aligned with a Facebook operation indicator may be thegeneration of a Facebook post. As another example, in an instance inwhich the object is aligned with the gallery operation indicator, allthe pictures and/or videos including the object may be listed on thedisplay so that the user can easily browse them. Additionally, upon anobject becoming aligned with an outlook operation indicator, anelectronic mail message may be generated and, in some instances one ormore history dialogues may be shown. Furthermore, in an instance inwhich the object becomes aligned with the social networking operationindicator, a listing of friends may be presented. As such, variousoperations may be performed depending upon the operation associated withor represented by the operation indicator with which the object becomesaligned.

Regardless of the operation that is associated with operation indicatorwith which the object becomes aligned, the operation that is performedis performed upon at least one aspect associated with the object, suchas based upon the name of the object, an appearance of the object,contacts associated with the object, metadata associated with the objector other aspects associated with the object. With respect to the SMS andFacebook operation indicators, a message or a post may be generated thatis directed to or otherwise associated with the person whose image hasbecome aligned with the operation indicator. By way of other examples,in an instance in which an image of a person is aligned with the galleryoperation indicator, all the pictures and/or videos including the personmay be listed to facilitate browsing. Additionally, upon an image of aperson becoming aligned with an outlook operation indicator, anelectronic mail message may be generated that is addressed to the personand, optionally, one or more history dialogues of message exchangesbetween the person and the user may be shown. Furthermore, in aninstance in which an image of a person becomes aligned with the socialnetworking operation indicator, a listing of friends that are common toboth the person and the user may be presented. Although several examplesare provided above, other types of operations may be performed basedupon at least one aspect associated with the object in otherembodiments.

In one embodiment, the performance of an operation associated with theoperation indicator is not only based upon a determination that theobject is aligned with the operation indicator, but also upon aselection of the object so as to avoid inadvertently performingoperations as the computing device is repositioned and the objectviewable via the display becomes aligned with various operationindicators. As such, the apparatus 10 may include means, such as theprocessor 12, the user interface 18 or the like, for providing forselection of the object. See block 56 of FIG. 4. The object may beselected in various manners. As shown in FIGS. 2 and 3, a cursor 34, 46may be caused to be presented upon the display. As such, the cursor maybe positioned upon the object and then selected by the user in order tocorrespondingly select the object. The cursor of this embodiment may berepositioned in various manners including by dragging and dropping thecursor via input provided via a touch screen display in the embodimentof FIG. 3 or via gestures, such as hand gestures that are captured by acamera 20 and interpreted by the processor 12 in conjunction with apass-through display of the type shown in FIG. 2. However, the objectmay be selected in still other manners including, for example, byaudible command, such as by saying the name of the person who has beenidentified as the object through facial recognition. In this embodiment,only once the object has been selected and a determination has been madethat the object is aligned with the operation indicator, the apparatus,such as the processor, may cause the operation to be performed upon atleast one aspect associated with the object.

The method, apparatus 10 and computer program product of an exampleembodiment of the present invention therefore facilitate the interactionby a user with an object that is viewable via a display, such asviewable through a pass-through display or viewable within an imagepresented upon a display of a mobile terminal 30. As such, a user maycause an operation to be performed that is based upon at least oneaspect associated with the object. The method, apparatus and computerprogram product of an example embodiment may therefore provide anintuitive mechanism for facilitating user interaction with an objectviewable via a display.

As described above, FIG. 4 illustrates a flowchart of an apparatus 10,method, and computer program product according to example embodiments ofthe invention. It will be understood that each block of the flowchart,and combinations of blocks in the flowchart, may be implemented byvarious means, such as hardware, firmware, processor, circuitry, and/orother devices associated with execution of software including one ormore computer program instructions. For example, one or more of theprocedures described above may be embodied by computer programinstructions. In this regard, the computer program instructions whichembody the procedures described above may be stored by a memory device14 of an apparatus employing an embodiment of the present invention andexecuted by a processor 12 of the apparatus. As will be appreciated, anysuch computer program instructions may be loaded onto a computer orother programmable apparatus (e.g., hardware) to produce a machine, suchthat the resulting computer or other programmable apparatus implementsthe functions specified in the flowchart blocks. These computer programinstructions may also be stored in a computer-readable memory that maydirect a computer or other programmable apparatus to function in aparticular manner, such that the instructions stored in thecomputer-readable memory produce an article of manufacture the executionof which implements the function specified in the flowchart blocks. Thecomputer program instructions may also be loaded onto a computer orother programmable apparatus to cause a series of operations to beperformed on the computer or other programmable apparatus to produce acomputer-implemented process such that the instructions which execute onthe computer or other programmable apparatus provide operations forimplementing the functions specified in the flowchart blocks.

Accordingly, blocks of the flowchart support combinations of means forperforming the specified functions and combinations of operations forperforming the specified functions for performing the specifiedfunctions. It will also be understood that one or more blocks of theflowchart, and combinations of blocks in the flowchart, can beimplemented by special purpose hardware-based computer systems whichperform the specified functions, or combinations of special purposehardware and computer instructions.

In some embodiments, certain ones of the operations above may bemodified or further amplified. Furthermore, in some embodiments,additional optional operations may be included, such as illustrated bythe blocks having a dashed outline in FIG. 4. Modifications, additions,or amplifications to the operations above may be performed in any orderand in any combination.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Moreover, although the foregoing descriptions and the associateddrawings describe example embodiments in the context of certain examplecombinations of elements and/or functions, it should be appreciated thatdifferent combinations of elements and/or functions may be provided byalternative embodiments without departing from the scope of the appendedclaims. In this regard, for example, different combinations of elementsand/or functions than those explicitly described above are alsocontemplated as may be set forth in some of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

What is claimed is:
 1. A method for identifying an object viewable via adisplay of a computing device, the method comprising: identifying theobject viewable via the display, wherein identification of the objectviewable via the display is performed by a processor; causing anoperation indicator to be presented upon the display concurrent with theobject, wherein the operation indicator is associated with an operationthat may be performed upon at least one aspect associated with theobject; determining, with the processor, that the object has becomealigned with the operation indicator as presented upon the display as aresult of movement of the display relative to the object, wherein arelative position of the operation indicator upon the display isindependent of the movement of the display relative to the object; andin response to determining that the object is aligned with the operationindicator, performing the operation associated with the operationindicator, wherein performing the operation comprises performing theoperation upon at least one aspect associated with the object.
 2. Amethod according to claim 1 further comprising providing for selectionof the object.
 3. A method according to claim 2 wherein performing theoperation comprises performing the operation only in an instance inwhich the object has been selected.
 4. A method according to claim 1wherein identifying an object comprises employing facial recognition toidentify a person.
 5. A method according to claim 1 wherein the displaycomprises a pass-through display, and wherein identifying an objectcomprises identifying an object viewable through the pass-throughdisplay.
 6. A method according to claim 1 further comprising: receivingan image that includes the object; and causing the image to be presentedupon the display prior to identifying the object.
 7. An apparatus foridentifying an object viewable via a display of a computing device, theapparatus comprising at least one processor and at least one memoryincluding computer program code, the at least one memory and thecomputer program code configured to, with the processor, cause theapparatus to at least: identify the object viewable via the display,wherein identification of the object viewable via the display isperformed by the apparatus including the at least one processor; causingan operation indicator to be presented upon the display concurrent withthe object, wherein the operation indicator is associated with anoperation that may be performed upon at least one aspect associated withthe object; determine that the object has become aligned with theoperation indicator as presented upon the display as a result ofmovement of the display relative to the object, wherein a relativeposition of the operation indicator upon the display is independent ofthe movement of the display relative to the object; and in response todetermining that the object is aligned with the operation indicator,perform the operation associated with the operation indicator, whereinthe at least one memory and the computer program code are configured to,with the processor, cause the apparatus to perform the operation byperforming the operation upon at least one aspect associated with theobject.
 8. An apparatus according to claim 7 wherein the at least onememory and the computer program code are further configured to, with theprocessor, cause the apparatus to provide for selection of the object.9. An apparatus according to claim 8 wherein the at least one memory andthe computer program code are configured to, with the processor, causethe apparatus to perform the operation by performing the operation onlyin an instance in which the object has been selected.
 10. An apparatusaccording to claim 7 wherein the at least one memory and the computerprogram code are configured to, with the processor, cause the apparatusto identify an object by employing facial recognition to identify aperson.
 11. An apparatus according to claim 7 wherein the displaycomprises a pass-through display, and wherein the at least one memoryand the computer program code are configured to, with the processor,cause the apparatus to identify an object by identifying an objectviewable through the pass-through display.
 12. An apparatus according toclaim 7 wherein the at least one memory and the computer program codeare further configured to, with the processor, cause the apparatus to:receive an image that includes the object; and cause the image to bepresented upon the display prior to identifying the object.
 13. Acomputer program product for identifying an object viewable via adisplay of a computing device, the computer program product comprisingat least one non-transitory computer-readable storage medium havingcomputer-executable program code portions stored therein, thecomputer-executable program code portions comprising program codeinstructions for: identifying the object viewable via the display,wherein identification of the object viewable via the display isperformed by the computing device during execution of thecomputer-executable program code portions; causing an operationindicator to be presented upon the display concurrent with the object,wherein the operation indicator is associated with an operation that maybe performed upon at least one aspect associated with the object;determining that the object has become aligned with the operationindicator as presented upon the display as a result of movement of thedisplay relative to the object, wherein a relative position of theoperation indicator upon the display is independent of the movement ofthe display relative to the object; and in response to determining thatthe object is aligned with the operation indicator, performing theoperation associated with the operation indicator, wherein performingthe operation comprises performing the operation upon at least oneaspect associated with the object.
 14. A computer program productaccording to claim 13 wherein the computer-executable program codeportions further comprise program code instructions for providing forselection of the object, and wherein the program code instructions forperforming the operation comprise program code instructions forperforming the operation only in an instance in which the object hasbeen selected.
 15. A computer program product according to claim 13wherein the program code instructions for identifying an object compriseprogram code instructions for employing facial recognition to identify aperson.
 16. A computer program product according to claim 13 wherein thedisplay comprises a pass-through display, and wherein the program codeinstructions for identifying an object comprise program codeinstructions for identifying an object viewable through the pass-throughdisplay.
 17. A computer program product according to claim 13 whereinthe computer-executable program code portions further comprise programcode instructions for: receiving an image that includes the object; andcausing the image to be presented upon the display prior to identifyingthe object.